我们提出蒙版频率建模（MFM），这是一种基于统一的基于频域的方法，用于自我监督的视觉模型预训练。在本文中，我们将视角转移到了频域中，而不是将蒙版令牌随机插入到空间域中的输入嵌入。具体而言，MFM首先掩盖了输入图像的一部分频率分量，然后预测频谱上的缺失频率。我们的关键见解是，由于沉重的空间冗余，预测频域中的屏蔽组件更理想地揭示了基础图像模式，而不是预测空间域中的掩盖斑块。我们的发现表明，通过对蒙版和预测策略的正确配置，高频组件中的结构信息和低频对应物中的低级统计信息都有用。 MFM首次证明，对于VIT和CNN，即使没有使用以下内容，简单的非叙事框架也可以学习有意义的表示形式：（i）额外的数据，（ii）额外的模型，（iii）蒙版令牌。与最近的蒙版图像建模方法相比，对成像网和几个鲁棒性基准的实验结果表明，MFM的竞争性能和高级鲁棒性。此外，我们还全面研究了从统一的频率角度来表示经典图像恢复任务对表示学习的有效性，并揭示了他们与MFM方法的有趣关系。项目页面：https：//www.mmlab-ntu.com/project/mfm/index.html。

肝脏的准确细分是诊断疾病的先决条件。自动分割是计算机辅助检测和肝病诊断的重要应用。近年来，医学图像的自动化处理已经取得了突破。然而，腹部扫描CT图像的低对比度和肝脏形态的复杂性使得精确的自动分割具有挑战性。在本文中，我们提出了RA V-NET，这是基于U-Net的改进的医学图像自动分割模型。它有以下三个主要创新。建议Cofres模块（复合原始功能剩余模块）。通过更复杂的卷积层和跳过连接，使其获得更高级别的图像特征提取功能并防止梯度消失或爆炸。建议AR模块（注意恢复模块）以减少模型的计算工作。另外，通过调整通道和LSTM卷积来感测编码和解码模块的数据像素之间的空间特征。最后，有效地保留了图像特征。介绍了CA模块（通道注意模块），用于提取具有依赖性的相关通道，并通过矩阵点产品加强它们，同时在没有依赖性的情况下削弱无关的通道。达到关注的目的。 LSTM卷积和CA模块提供的注意机制是强证神经网络性能的保证。 U-Net网络的准确性：0.9862，精确度：0.9118，DSC：0.8547，JSC：0.82。 RA V-NET的评估指标，精度：0.9968，精确度：0.9597，DSC：0.9654，JSC：0.9414。分割效果的最代表性度量是DSC，其在U-NET上改善0.1107，JSC改善0.1214。

对比的自我监督学习在很大程度上缩小了对想象成的预先训练的差距。然而，它的成功高度依赖于想象成的以对象形象，即相同图像的不同增强视图对应于相同的对象。当预先训练在具有许多物体的更复杂的场景图像上，如此重种策划约束会立即不可行。为了克服这一限制，我们介绍了对象级表示学习（ORL），这是一个新的自我监督的学习框架迈向场景图像。我们的主要洞察力是利用图像级自我监督的预培训作为发现对象级语义对应之前的，从而实现了从场景图像中学习的对象级表示。对Coco的广泛实验表明，ORL显着提高了自我监督学习在场景图像上的性能，甚至超过了在几个下游任务上的监督Imagenet预训练。此外，当可用更加解标的场景图像时，ORL提高了下游性能，证明其在野外利用未标记数据的巨大潜力。我们希望我们的方法可以激励未来的研究从场景数据的更多通用无人监督的代表。

对比度学习最近在无监督的视觉表示学习中显示出巨大的潜力。在此轨道中的现有研究主要集中于图像内不变性学习。学习通常使用丰富的图像内变换来构建正对，然后使用对比度损失最大化一致性。相反，相互影响不变性的优点仍然少得多。利用图像间不变性的一个主要障碍是，尚不清楚如何可靠地构建图像间的正对，并进一步从它们中获得有效的监督，因为没有配对注释可用。在这项工作中，我们提出了一项全面的实证研究，以更好地了解从三个主要组成部分的形象间不变性学习的作用：伪标签维护，采样策略和决策边界设计。为了促进这项研究，我们引入了一个统一的通用框架，该框架支持无监督的内部和间形内不变性学习的整合。通过精心设计的比较和分析，揭示了多个有价值的观察结果：1）在线标签收敛速度比离线标签更快； 2）半硬性样品比硬否定样品更可靠和公正； 3）一个不太严格的决策边界更有利于形象间的不变性学习。借助所有获得的食谱，我们的最终模型（即InterCLR）对多个标准基准测试的最先进的内图内不变性学习方法表现出一致的改进。我们希望这项工作将为设计有效的无监督间歇性不变性学习提供有用的经验。代码：https：//github.com/open-mmlab/mmselfsup。

Driven by improved architectures and better representation learning frameworks, the field of visual recognition has enjoyed rapid modernization and performance boost in the early 2020s. For example, modern ConvNets, represented by ConvNeXt, have demonstrated strong performance in various scenarios. While these models were originally designed for supervised learning with ImageNet labels, they can also potentially benefit from self-supervised learning techniques such as masked autoencoders (MAE). However, we found that simply combining these two approaches leads to subpar performance. In this paper, we propose a fully convolutional masked autoencoder framework and a new Global Response Normalization (GRN) layer that can be added to the ConvNeXt architecture to enhance inter-channel feature competition. This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition benchmarks, including ImageNet classification, COCO detection, and ADE20K segmentation. We also provide pre-trained ConvNeXt V2 models of various sizes, ranging from an efficient 3.7M-parameter Atto model with 76.7% top-1 accuracy on ImageNet, to a 650M Huge model that achieves a state-of-the-art 88.9% accuracy using only public training data.

A step-search sequential quadratic programming method is proposed for solving nonlinear equality constrained stochastic optimization problems. It is assumed that constraint function values and derivatives are available, but only stochastic approximations of the objective function and its associated derivatives can be computed via inexact probabilistic zeroth- and first-order oracles. Under reasonable assumptions, a high-probability bound on the iteration complexity of the algorithm to approximate first-order stationarity is derived. Numerical results on standard nonlinear optimization test problems illustrate the advantages and limitations of our proposed method.

Masked image modeling (MIM) has shown great promise for self-supervised learning (SSL) yet been criticized for learning inefficiency. We believe the insufficient utilization of training signals should be responsible. To alleviate this issue, we introduce a conceptually simple yet learning-efficient MIM training scheme, termed Disjoint Masking with Joint Distillation (DMJD). For disjoint masking (DM), we sequentially sample multiple masked views per image in a mini-batch with the disjoint regulation to raise the usage of tokens for reconstruction in each image while keeping the masking rate of each view. For joint distillation (JD), we adopt a dual branch architecture to respectively predict invisible (masked) and visible (unmasked) tokens with superior learning targets. Rooting in orthogonal perspectives for training efficiency improvement, DM and JD cooperatively accelerate the training convergence yet not sacrificing the model generalization ability. Concretely, DM can train ViT with half of the effective training epochs (3.7 times less time-consuming) to report competitive performance. With JD, our DMJD clearly improves the linear probing classification accuracy over ConvMAE by 5.8%. On fine-grained downstream tasks like semantic segmentation, object detection, etc., our DMJD also presents superior generalization compared with state-of-the-art SSL methods. The code and model will be made public at https://github.com/mx-mark/DMJD.

Considering the computation complexity, we propose a Guided Hybrid Quantization with One-to-one Self-Teaching (GHOST}) framework. More concretely, we first design a structure called guided quantization self-distillation (GQSD), which is an innovative idea for realizing lightweight through the synergy of quantization and distillation. The training process of the quantization model is guided by its full-precision model, which is time-saving and cost-saving without preparing a huge pre-trained model in advance. Second, we put forward a hybrid quantization (HQ) module to obtain the optimal bit width automatically under a constrained condition where a threshold for distribution distance between the center and samples is applied in the weight value search space. Third, in order to improve information transformation, we propose a one-to-one self-teaching (OST) module to give the student network a ability of self-judgment. A switch control machine (SCM) builds a bridge between the student network and teacher network in the same location to help the teacher to reduce wrong guidance and impart vital knowledge to the student. This distillation method allows a model to learn from itself and gain substantial improvement without any additional supervision. Extensive experiments on a multimodal dataset (VEDAI) and single-modality datasets (DOTA, NWPU, and DIOR) show that object detection based on GHOST outperforms the existing detectors. The tiny parameters (<9.7 MB) and Bit-Operations (BOPs) (<2158 G) compared with any remote sensing-based, lightweight or distillation-based algorithms demonstrate the superiority in the lightweight design domain. Our code and model will be released at https://github.com/icey-zhang/GHOST.

Automatic font generation without human experts is a practical and significant problem, especially for some languages that consist of a large number of characters. Existing methods for font generation are often in supervised learning. They require a large number of paired data, which are labor-intensive and expensive to collect. In contrast, common unsupervised image-to-image translation methods are not applicable to font generation, as they often define style as the set of textures and colors. In this work, we propose a robust deformable generative network for unsupervised font generation (abbreviated as DGFont++). We introduce a feature deformation skip connection (FDSC) to learn local patterns and geometric transformations between fonts. The FDSC predicts pairs of displacement maps and employs the predicted maps to apply deformable convolution to the low-level content feature maps. The outputs of FDSC are fed into a mixer to generate final results. Moreover, we introduce contrastive self-supervised learning to learn a robust style representation for fonts by understanding the similarity and dissimilarities of fonts. To distinguish different styles, we train our model with a multi-task discriminator, which ensures that each style can be discriminated independently. In addition to adversarial loss, another two reconstruction losses are adopted to constrain the domain-invariant characteristics between generated images and content images. Taking advantage of FDSC and the adopted loss functions, our model is able to maintain spatial information and generates high-quality character images in an unsupervised manner. Experiments demonstrate that our model is able to generate character images of higher quality than state-of-the-art methods.

Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.